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The  Need  for  a  Revolution  in 
Assessment  Capability 


The  calm  that  followed  the  Cold  War  is  over.  Today's  war-torn  world  is  wracked  by 
discontinuous  change  and  intense  struggle  over  cultural  values  and  regional  power. 
Rogue  states,  violent  extremism,  and  devious  threat  tactics  combine  with  potential 
access  to  weapons  of  mass  destruction  to  pose  grave  and  unprecedented  challenges. 
Security  has  regained  its  pre-eminence  among  national  policy  concerns,  and 
intelligence,  as  always,  must  provide  the  first  line  of  forward  defense. 

However,  notwithstanding  substantial  progress  in  recent  years,  the  United  States 
Intelligence  Community  (IC)  is  not  yet  performing  up  to  its  full  state-of-the-art 
potential.  Much  of  official  and  influential  America  —  the  legislative  and  executive 
branches  of  the  federal  government,  historians,  defense  analysts,  and  the  informed 
public  -  have  come  increasingly  to  the  conclusion  that  the  IC  suffers  from  systemic 
deficiencies  that  must  be  rectified  if  warning  is  to  improve. 

In  addition  to  the  age-old  difficulties  of  penetrating  denied  areas  and  ideological 
movements,  we  also  lack  the  sophisticated  analysis  and  assessment  capability  required 
to  convert  the  extensive  information  we  do  collect  into  prediction  and  warning 
sufficiently  powerful  and  timely  to  prevent  intelligence  failure  and  surprise.  This 
incapacity  in  part  reflects  the  real  difficulties  of  intelligence  -  but  only  in  part.  It  also 
reflects  deep-seated  institutional  culture  and  venerable  but  increasingly  obsolete  beliefs 
about  the  supposedly  insuperable  analytic  limitations  intelligence  organizations  face. 

This  essay  argues  that  intelligence  assessment  could  achieve  substantial 
predictive  power,  to  a  degree  that  greatly  exceeds  today's  standards.  It  will  seek  to 
demonstrate  both  the  need  for  such  a  revolution  in  capability  and  the  feasibility  of 
achieving  it,  and  will  outline  a  strategic  program  plan  to  do  so.  This  proposal  is 
designed  to  implement  core  elements  of  the  National  Intelligence  Strategy,  the  2004 
Intelligence  Reform  Act,  and  the  recommendations  of  the  9/11  Commission  Report 
and  related  intelligence  reform  initiatives  and  studies^. 


1  The  National  Intelligence  Strategy  of  the  United  States  of  America:  Transformation  through  Innovation  and 
Integration:  Office  of  the  Director  of  National  Intelligence,  October  2005.  National  Intelligence  Reform  Act 
of  2004  (S.2845, 108*  Congress,  6  October  2004).  The  9/11  Commission  Report:  Final  Report  of  the  National 
Commission  on  Terrorist  Attacks  Upon  the  United  States,  Washington,  D.C.,  2004;  W.W.  Norton  &  Co.,  or 
http:/  / www.9-llcommission.gov/ report/ index.htm.  Report  to  the  President  of  the  Commission  on  the 
Intelligence  Capabilities  of  the  United  States  Regarding  Weapons  of  Mass  Destruction,  Washington,  DC,  2005; 
http:  /  / www.wmd.gov/ report/ index.html.  Two  unofficial  reform  studies  also  were  important 
influences  mspirmg  and  shaping  this  proposal:  Waltz,  Edward:  Knowledge  Management  in  the  Intelligence 
Enterprise;  Artech  House,  2003;  and  Barger,  Deborah  G.:  Toward  a  Revolution  in  Intelligence  Affairs;  RAND 
Technical  Report  TR-242-CMS,  2005,  p.  47;  http:/ / www.rand.org/ publications/TR/TR242/. 
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strategic  Vision  for  Analytic  Transformation 


'Predictive  network-centric  intelligence  assessment  offers  the  ability  to  foresee 
and  warn  of  threats,  be  these  strategic  or  tactical  and  conventional  or  asymmetric,  and 
either  forestall  or  stay  abreast  of  rapidly  breaking  crises  and  high-speed  conflict 
operations.  Such  enhanced  anticipatory  power  and  adaptivity  could  be  achieved 
through  improved  analysis  and  assessment  based  on  current  technology  and  today's 
collection  capabilities.  The  required  methodology  and  business  process  involve  a 
structured  but  adaptive  set  of  procedures  involving  interagency  collaboration, 
alternative  analyses,  and  the  rigorous  testing  of  assumptions  and  assessments.  All  three 
of  these  elements  -  community-wide  collaboration,  expansive  creative  imagination,  and 
rigorous  critical  reasoning  —  are  equally  vital.  In  fact,  they  are  interdependent:  a 
quantum  leap  in  predictive  power  could  come,  and  could  only  come,  as  a  synergetic 
effect  of  their  dynamic  integration. 

Two  basic  premises  underlie  this  proposal.  First,  we  trust  that  promulgated  ODNI 
policy  on  IC  collaboration  and  information  sharing  (e.g.,  relaxed  need-to-know 
constraints)  will  ensure  that  web-enabled  virtual  collaboration  (e.g.,  analysts'  'wikis' 
and  'blogs')  will  soon  become  the  normal  operating  mode  for  analysts  sitting  at  their 
desks  producing  daily  intelligence  across  the  community.  Such  baseline  collaboration  is 
necessary.  The  second  premise,  however,  holds  that  in  and  of  itself  such  distributed 
collaboration  will  not  maximize  the  IC's  predictive  potential.  Much  more  can  and  must 
be  done.  The  IC  must  also  implement  powerful  new  capabilities  for  proximate, 
'synchronous'  collaboration,  and  then  build  from  both  of  these  collaborative  advances 
a  radical  new  capability  for  IC-wide  collaboration  that  exceeds  any  plans  yet 
promulgated. 

The  visible  centerpiece  of  this  transformed  capability  would  lie  in  the  creation  of  a 
community-wide  system  of  advanced  intelligence  assessment  laboratories  that  would 
operate  according  to  a  formal,  documented  assessment  methodology  -  specifically, 
scientific  methodology^.  Within  these  labs  analytic  project  teams  would  use  a  wide 
array  of  both  interdisciplinary  and  problem-specific  methods,  tools  and  techniques  to 
produce  all-source  finished  intelligence,  while  simultaneously  advancing  the  state  of 
the  analytic  art  through  experience  and  lessons-learned.  A  suitable  term  for  such 
'knowledge  factories'  might  be  'computational  collaboratories'. 

The  premier  IC  computational  collaboratory  should  be  controlled  directly  by  the 
ODNI,  support  the  national  command  authority  (NCA),  and  serve  as  the  main  engine 
for  fostering  interagency  teaming  on  high-level  intelligence  projects  such  as  National 


2  A  seminal  study  by  Isaac  Ben-Israel  recommended  the  application  of  scientific  methodology  to 
intelligence  assessment:  Ben-Israel,  Isaac:  "Philosophy  and  Methodology  of  Intelligence:  the  Logic  of  the 
Estimate  Process";  Intelligence  and  National  Security,  Vol.  4,  No.  4,  October  1989,  pp.  660-718.  Two 
previous  Galileo  papers  also  have  called  for  the  application  of  scientific  method  to  intelligence 
assessment:  Bruce,  James:  "Dynamic  Adaptation:  a  Twenty-First  Century  Intelligence  Paradigm"  (2004); 
and  Pesce,  Joseph  and  Randolph  Tauss:  "Science  and  Intelligence  Analysis:  the  Requirement  for  'Critical 
Thinking'  Training  in  the  Intelligence  Community"  (2004). 
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Intelligence  Estimates  (NIEs).^  Eventually  the  IC  should  be  netted  together  by  a  multi¬ 
agency  complex  of  such  laboratories  -  a  'system  of  systems'  —  one  in  each  intelligence 
agency  (all-source  and  single-source),  all  integrated  over  a  wide-area  network  at 
multiple  levels  of  security.  The  result  would  be  a  'boundaryless'  architecture  of 
permanent,  pervasive  collaboration.^ 

Implementing  this  proposal  would  both  require  and  propel  a  revolution  in  both  the 
synthetic  (creative-imaginative)  and  analytic  (rigorous-critical)  dimensions  of 
intelligence  assessment,  as  part  of  a  major  advance  in  the  IC's  'analytic  culture'.^  Yet 
since  it  is  methodological,  managerial  and  behavioral  in  nature,  this  solution  would 
require  only  modest  expenditure  in  technical  systems  development  and  acquisition, 
infrastructure,  or  manning  levels.  The  new  methodology  would  supplement  today's 
standard  operating  procedure,  which,  although  practical  and  productive,  relies  on  a 
risky  combination  of  practices:  routinized  bureaucratic  procedure  conjoined  with 
informal,  intuitive  reasoning.  Both  of  these  exacerbate  susceptibility  to  internal  bias 
and  adversary  deception,  and  hence  increase  vulnerability  to  intelligence  failure  and 
surprise.^ 

In  sum,  then,  IC  threat  anticipation  and  warning  could  be  improved  substantially, 
affordably,  and  relatively  quickly,  should  the  national  leadership  act  to  transform  the 
methodology  and  systemic  processes  governing  intelligence  synthesis,  analysis  and 
assessment.  Moreover,  beyond  its  immediate  value  in  improving  assessment  within  the 
disparate  agencies  of  the  national  IC,  the  plan  proposed  herein  would  both  strengthen 
and  integrate  the  community,  accelerate  the  intelligence  cycle  rate  (the  'optempo'  of  our 
'OODA  Loop'),^  and  transform  the  IC  into  a  unified  system  of  agile  learning 
organizations  with  a  network-centric  capability  optimally  adapted  to  today's  security 
challenges  and  intellectual  and  technological  opportunities. 


3  Pursuant  to  Sections  123,  National  Intelligence  Council;  145,  Office  of  Alternative  Analysis;  and  222, 
Independence  of  Intelligence,  in  the  2004  National  Intelligence  Reform  Act. 

'Boundarylessness'  -  in  effect,  horizontal  integration  -  was  one  of  fhe  core  fenets  of  CEO  Jack  Welch's 
leadership  of  General  Elecfric,  a  conglomerafe  nof  unlike  fhe  IC,  in  fhe  1980s-90s.  Welch,  Jack,  with  John 
A.  Byrne:  Jack:  Straight  From  the  Gut:  Warner  Business  Books,  2001. 

5  On  'analytic  culture',  see  Johnston,  Rob:  Analytic  Culture  in  the  U.S.  Intelligence  Community:  an 
Ethnographic  Study,  Central  Intelligence  Agency  Center  for  fhe  Sfudy  of  Intelligence  (CIA/ CSI),  2005. 

6  A  burgeoning  multidisciplinary  literature  addresses  the  problems  and  causes  of  intelligence  failure. 
Cognitive  psychology  has  examined  the  problems  of  bias  in  intuifive  reasoning.  The  seminal  application 
to  intelligence  is  Heuer,  Richards  J.,  Psychology  of  Intelligence  Analysis;  CIA/  CSI,  1999.  Seminal  studies  of 
group  and  governmental  error  include  Janis,  Irving  L.:  Groupthink:  Psychological  Studies  of  Policy  Decisions 
and  Fiascoes;  Houghton-Mifflin,  1972/1982,  and  Jervis,  Robert:  Perception  and  Misperception  in  International 
Politics,  Princeton  University  Press,  1976.  Related  challenges  in  the  practice  of  science  are  addressed  in 
Kuhn,  Thomas,  The  Structure  of  Scientific  Revolutions,  Universify  of  Chicago  Press,  1972.  The  historical 
literature  on  intelligence  failure  is  growing  as  well;  a  classic  case  study  is  Wohlstetter,  Roberta:  Pearl 
Harbor:  Warning  and  Decision;  Stanford  University  Press,  1962. 

^  Terminology  from  milifary  command  and  confrol  theory:  operational  tempo  or  'optempo'  refers  fo  fhe 
cyclic  rafe  of  C4ISR  fask  accomplishment  within  and  among  units.  The  Observation-Orientation- 
Decision-Action  (OODA)  Loop  was  Col.  John  Boyd's  generalization  of  what  we  call  the  'intelligence 
cycle'  or  'TPED':  Tasking,  Processing,  Exploitation  and  Dissemination. 
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The  Problem:  Meeting  the  Challenge  of  Foresight, 
Anticipation  and  Warning 

Traditional  1C  Assessment  Culture  and  Practice 

'Methodology'  refers  to  the  logic  governing  methods.  It  derives  from  an  underlying 
philosophy,  particularly  in  epistemology,  the  theory  of  knowledge.  Methodology  is 
applied  epistemology  and  takes  the  form  of  an  integrated  theory  of  assessment  that 
specifies  the  domain  of  application,  the  criteria  of  truth,  the  definition  of  evidence,  and 
the  rules  for  inference.  Thus  although  tools  and  techniques  and  even  fairly  basic 
methods  proliferate  in  abundance,  all  of  these  fall  under  a  very  small  number  of 
underlying  methodologies. 

Essentially  two  logical-factual  methodologies  exist  for  understanding  the  world  of 
physical  things  and  human  behavior,  one  involving  informal,  and  the  other  formal, 
methods.  The  informal  methodology  can  be  characterized  broadly  as  historiography, 
the  more  or  less  scholarly  study  of  qualitative  evidence  formulated  as  words  and 
sentences  in  natural  language.  This  typically  is  undertaken  by  individual  scholars  or 
analysts,  working  independently  or  compartmentalized  under  vertical  chains  of 
authority  ('stovepipes')  in  offices  within  bureaucracies.  Traditional  intelligence 
assessment  methodology  has  always  been  historiographical. 

By  definition,  historiography  is  strictly  descriptive  ('graphic').  It  can  be  applied  to 
two  timeframes,  both  of  them  retrospective:  the  long-term  or  distant  past,  this  being 
history  proper  (in  intelligence,  this  is  'term  analysis');  or  the  short-term  or  recent  past, 
this  being  journalism  or  current  intelligence.  By  its  very  nature  historiography  cannot 
produce  reliable  inferences  or  assessments  concerning  the  other  important  timeframe: 
the  future.  Historiography  cannot  predict. 

The  risks  associated  with  the  misapplication  of  informal,  intuitive,  retrospective 
methods  to  problems  of  prediction  are  becoming  increasingly  well  understood.  A 
burgeoning  literature  has  emerged  in  cognitive  and  social  psychology,  perceptions 
theory  and  the  history  of  science  that  describes  theory  and  findings  concerning  human 
perception,  cognitive  'heuristics'  (habituated  rules  of  thumb),  mental  and  institutional 
'paradigms'  (belief  systems)  and  'satisficing'  (the  tendency  to  opt  for  quick,  seemingly 
adequate  solutions  rather  than  search  exhaustively  for  utility-maximizing  solutions^). 
Herbert  Simon,  Daniel  Kahneman  and  Amos  Tversky,  Robert  Jervis,  Irving  Janis  and 


*  'Satisfice':  a  combination  of  'satisfy'  and  'suffice'.  Distinguished  from  classical  utility  maximization. 
Concept  developed  in  Simon,  Herbert  A.:  "A  Behavioral  Model  of  Rational  Choice",  Quarterly  Journal  of 
Economics,  Vol.  LXIX,  February  1955,  pp.  99-101  (reprinted  in  Simon,  Models  of  Man:  Social  and  Rational; 
1957).  Note  that  satisficing  often  optimizes  under  conditions  of  constraint  (especially  temporal)  and  can 
therefore  often  provide  more  rational  (utility-maximizing)  outcomes  than  can  'paralysis  by  analysis'. 
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Thomas  Kuhn  are  pre-eminent  names  in  these  fields^.  Satisficing  heuristics  employ 
assumption-based  reasoning  for  the  sake  of  simplicity  and  efficiency,  and  are  useful 
under  normal  circumstances  of  low  risk.  They  often  are  optimal,  in  fact,  especially 
under  constraints  such  as  time  pressure  (as  General  George  S.  Patton  noted,  "A  good 
plan  today  is  better  than  a  perfect  plan  next  week").  Nonetheless,  haste  and  habit  have 
a  cost  in  introducing  patterns  of  bias  and  error  into  human  reasoning.  Moreover,  they 
are  highly  susceptible  to  deception  and  surprise. 

Groups  and  organizations  can  fall  prey  to  similar  decision  maladies,  which  can  be 
summarized  broadly  under  the  rubric  'groupthink',  the  process  whereby  group 
members  conform  to  given  assumptions,  stifling  doubt  and  debate  so  as  not  to  disrupt 
group  togetherness.^^  DIA  Indications  and  Warning  (I&W)  expert  Cynthia  Grabo  notes 
that  "The  rejection  of  evidence  incompatible  with  one's  own  hypotheses  or 
preconceptions,  the  refusal  to  accept  or  to  believe  that  which  is  unpleasant  or  disturbing 
or  which  might  upset  one's  superiors  -  these  are  far  more  common  failings  than  most 
people  suspect". 

The  distinguished  Richards  J.  Heuer,  long  the  leading  counter-deception  analyst  and 
trainer  in  the  CIA,  summarized  the  findings  from  cognitive  psychology  and  applied 
them  to  the  intelligence  problem  in  his  aforementioned  Psychology  of  Intelligence 
Analysis.  The  literature  on  intelligence  and  especially  on  deception  and  intelligence 
failure  amplify  these  findingsi^. 

In  sum,  then,  by  their  inherent  nature,  qualitative  methodologies  such  as 
historiography  are  incapable  of  prediction  and  entail  an  excessively  high  risk  of  error 
when  employed  for  that  purpose  -  which  is  precisely  what  traditional  intelligence  does, 
in  the  U.S.  and  other  countries.  The  inevitable  result,  seen  time  and  again,  decade  in 
and  decade  out,  is  an  unnecessarily  high  rate  of  intelligence  failure  and  surprise,  with 
no  discernable  trend  line  of  upward  improvement  so  far. 

What  then  is  required  for  effective  anticipation  and  warning? 

Prediction:  Process  and  Prerequisites 

CIA  has  defined  intelligence  in  relevant  terms:  "Reduced  to  its  simplest  terms, 
intelligence  is  knowledge  and  foreknowledge  of  the  world  around  us  -  the  prelude  to 


®  Daniel  Kahneman  shared  the  2002  Nobel  Prize  in  Economics  for  his  work  in  cognitive  psychology.  The 
multidisciplinary  literature  here  is  vast  and  indispensable.  Space  constraints  restrict  full  citation,  but  oft- 
cited  authors  include  Herbert  A.  Simon,  Daniel  Kahnemarm,  Amos  Tversky,  Irving  L.  Janis,  Allen  Newell, 
Robert  Jervis,  Richard  E.  Nisbett,  Lee  Ross,  Thomas  Gilovich,  Scott  Plouse  and  Thomas  Kuhn  (op.  cit.,  fn  3 
above).. 

10  Janis,  Groupthink,  op.  cit. 

11  Grabo,  Cynthia:  Anticipating  Surprise:  Analysis  for  Strategic  Warning;  Joint  Military  Intelligence  College, 
Washington,  DC,  2002,  p.  39. 

12  The  literature  here  as  well  is  substantial,  but  theory  remains  underdeveloped.  Oft-cited  authors  include 
Richard  Betts,  Michael  Handel,  Ephraim  Kam,  James  J.  Wirtz,  Barton  Whaley,  Donald  C.  Daniel  & 
Katherine  L.  Herbig,  John  Gooch  &  Amos  Perlmutter,  Michael  Howard,  Roy  Godson,  Roger  Hesketh  & 
Nigel  West,  and  Thaddeus  Holt. 
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decisions  and  action  by  U.S.  policymakers" Foreknowledge,  conceived  in  a  scientific, 
business,  or  intelligence  sense,  obviously  does  not  imply  psychic  clairvoyance  or 
deterministic  'point'  prediction.  On  the  contrary,  it  refers  to  rational  expectation; 
estimative  forecasting  based  on  available  evidence  and  formal  analytic  methods. 

Such  estimative  prediction  requires  both  creative  and  critical  thinking,  specifically: 
(1)  formal  analytic  rigor  and  the  explication  of  assumptions,  and  (2)  perception  and 
understanding  across  both  the  depth  and  the  breadth  of  a  problem  domain,  each  of  which 
imposes  its  own  specific  methodological  and  practical  requirements.  First  we  will 
examine  the  problems  of  depth  and  breadth  and  then  address  that  of  rigor. 

Critical  Thinking  and  Analytic  Depth 

Extension  in  depth  consists  of  analysis  and  extrapolation,  the  projection  of  patterns 
and  trends  into  the  long-  or  short-term  future  based  on  statistics  and  mathematical 
probability.  Such  trend- 
projection  is  typical  in  the 
related  group  of  disciplines 
known  as  neo-classical  eco¬ 
nomics,  operations  research, 
management  science,  and  de¬ 
cision  theory.  It  relies  on  strict 
deductive  inference  from 
given  premises,  including  the 
precept  that  man  is  a  utility- 
maximizing  'rational  actor' 

('economic  man').  Moreover, 

'man'  the  'actor'  has  always 
explicitly  included  both  the 
subject  of  analysis  (e.g.,  a 
foreign  decision-maker)  and, 
implicitly,  the  subject  perform¬ 
ing  the  analysis  (the  analyst). 

Such  methods,  although  they 
provide  powerful  tools  for 
testing,  eliminating  or  justifying  propositions,  are  only  as  good  as  their  assumptions  — 
the  old  'garbage  in,  garbage  out'  or  'GIGO'  problem.  Yet,  as  noted  above,  social 
scientists  and  philosophers  have  revealed  limitations  and  sources  of  error  in  human 
reasoning  that  make  the  selection  of  assumptions  a  'non-trivial'  task  subject  to  human 
error.  These  patterns  of  error  not  only  cripple  qualitative  methodology,  they  also  afflict 
strictly  analytico-deductive  methods  as  well,  in  the  following  ways. 


NOTE  on  TERMINOLOGY: 

Understanding  intelligence  assessment  requires  clear 
distinction  among  related  terms  such  as  methodology  and 
methods;  analysis,  synthesis,  and  assessment  or 
judgement;  and  induction,  abduction  and  deduction. 

Briefly,  methodology  is  the  logic  governing  methods. 
Synthesis  builds  order  and  unity  out  of  disorder  -  in  this 
case,  theory  and  models  out  of  sparse  or  massive  data. 
It  encompasses  induction,  the  collection  and  ordering  of 
empirical  data,  and  abduction,  the  generation  and 
selection  of  hypotheses  for  testing.  Hypothesis  testing 
in  turn  involves  deductive  analysis,  the  isolation  of 
variables,  definition  of  metrics,  and  determination  of 
logical-mathematical  relationships  among  these.  Re¬ 
integration  and  final  assessment  again  involves 
synthesis,  but  at  a  higher  level  (completing  a  cycle  of  the 
spiral).  Synthesis  and  analysis  are  complementary 
opposites,  being  jointly  necessary  but  individually 
insufficient  for  sound  assessment.  Finally,  the  term 
assessment  refers  herein  to  the  entire  integrated 
process  of  synthesis-analysis-synthesis. 


13  "A  Consumer's  Guide  to  Intelligence";  CIA  Office  of  Public  Affairs,  1999,  p.  vii. 
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First,  human  rationality  is  'bounded';  limited  and  often  biased  by  psychological 
factors  involving  perception,  cognition,  emotion,  values  and  personal  interestd"^  Busy 
observers  and  participants  often  tend  to  form  images  and  frame  opinions  rather  hastily 
and  uncritically  ('leaping  to  conclusions'  through  'hasty  generalization').  These 
perceptions  produce  often  implicit  and  inchoate  mental  images  of  the  world.  This 
subjectivity  and  tacit  imprecision  appear  to  be  'co-dependent',  each  'enabling'  the  other; 
comfortable  biases  are  best  defended  if  left  camouflaged,  which  in  turn  protects  them 
even  when  they're  not  a  source  of  comfort. 

Next,  once  observers  have  formed  images  and  framed  opinions,  they  then  tend  to 
internalize  them,  identify  with  them,  personalize  them,  and  defend  them  against  all 
incoming  data  and  criticism.  Thus  even  sound  assumptions  persist  and  can  be 
rendered  obsolete  by  dynamic  change  in  the  environment. 

When  they  rest  on  such  dubious  premises,  even  the  most  powerful  analytico- 
deductive  methods  lack  the  ability  not  only  to  predict  non-linear,  revolutionary 
discontinuities,  but  even  to  identify  extant  and  emerging  trends  well  enough  reliably  to 
predict  even  evolutionary  change.  Until  advanced  models  emerge^^,  the  intelligence 
community  can  prevent  surprise  (or  at  least  reduce  the  risk,  rate  and  severity  of 
surprise)  only  if  it  can  anticipate  the  innovative,  asymmetric,  and  often-devious  gambits 
an  adversary  might  employ.  This  requires  a  very  wide  span  of  peripheral  vision  that 
can  imagine  not  just  conventional  contingencies,  but  plausible  unprecedented  ones  as 
well  -  hypotheses  and  alternative  scenarios.  This  breadth  of  imagination  is  a  synthetic 
rather  than  an  analytic  function.  It  is  what  enables  us  to  'ask  the  right  questions'  in  the 
first  place. 

Creative  Thinking  and  Synthetic  Breadth 

In  surprises,  the  'ball  comes  in  out  of  left  field',  outside  the  victim's  span  of 
'peripheral  vision'  at  the  time  of  the  event,  which  often  is  narrowly  focused  in  a  given 
direction.  Surprise  thus  usually  comes  in  the  form  of  events  for  which  there  had  been 
previous  evidence  that  was  ignored  or  dismissed.  In  Grabo's  words;  "It  is  the  history  of 
every  great  warning  crisis  that  the  post-mortems  have  turned  up  numerous  relevant 
facts  or  pieces  of  information  which  were  available  but  which,  for  one  reason  or 
another,  were  not  considered  in  making  assessments  at  the  time".^^  As  noted  above, 
social  scientists  have  explored  some  of  the  reasons  for  this  misdirection  of  attention. 


Simon,  Herbert  A.;  op.  cit. 

15  These  would  have  to  take  the  form  of  complex  adaptive  systems  (CAS)  models  that  produce  self- 
organized  emergence  and  non-linear  dynamics.  The  most  promising  models  available  today  appear  to  be 
those  based  on  the  work  of  Jay  Forrester  (see,  inter  alia,  Principles  of  Systems,  Pegasus  Communications, 
1968/1990,  and  http:/ /www.systemdynamics.org/).  Highly  sophisticated  CAS  models  of  emergence 
and  evolutionary  dynamics  (e.g.,  agent-based  models)  promise  even  more  powerful  capabilities  for  the 
future.  The  Santa  Fe  Institute  (SFl;  http:/ /www.santafe.edu/)  and  New  England  Complex  Systems 
Institute  (NECSl;  http:/ / necsi.org/1  are  good  sources  for  relevant  research  information. 

15  Grabo,  op.  cit.,  p.  9.  This  thesis  underlay  much  of  Wohlstetter's  opus  (op.  cit.);  she  indicated  that  much 
data  that  appears  retrospectively  to  have  constituted  clear  l&W  ('signals',  in  information  theoretic  terms) 
was  lost  in  the  surrounding  'noise'.  While  retrospective  criticism  can  be  disparaged  as  '20/20  hindsight'. 
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However,  no  rules-governed  procedure,  algorithm  or  software  exists  for  ensuring 
perspicacity,  imagination,  creativity,  and  discovery  'on  demand'.  These  are  stochastic, 
wholistic  and  'emergent'  complex-systems  phenomena,  thriving  through  'self- 
organized  criticality  at  the  edge  of  order  and  chaos'. Deterministic,  reductionistic 
analytics  do  not  promote  or  proliferate  hypotheses:  on  the  contrary,  as  a  'convergent' 
mode  of  reasoning,  analysis  narrows  focus  and  eliminates  hypotheses.  Hypothesis 
generation  is  the  'art'  of  science,  the  domain  where  intuition  and  imagination  can  and 
must  play  an  indispensable  role  in  science  and  decision-making.  What  is  required  is  a 
reliable  method  for  broadening  such  'divergent'  reasoning  (a  process  the  IC 
misleadingly  refers  to  as  alternative  'analysis').  Much  of  the  rest  of  the  paper  proposes 
a  solution  to  the  problem  of  promoting  these  synthetic  methods.^^ 

The  underlying  prerequisites  lie  in  induction  and  'abduction':  the  collection, 
collation,  and  ordering  of  data,  and  the  process  of  'pattern  recognition'  required  to  infer 
hypotheses  to  explain  the  data  and  make  predictions  based  on  it.^^  Once  these 
challenges  are  solved,  proper  task  sequencing  requires  that  this  synthetic  phase  precede 
the  deductive  analyses  described  above,  for  it  is  here  where  alternative  hypotheses 
emerge  for  subsequent  testing  using  the  formal  analytic  methods. 

Analytico-Synthetic  Integration 

Sound  forecasting  requires  that  both  elements  be  integrated  into  a  single  unified 
methodology  in  order  to  ensure  that  the  full  range  of  potential  contingencies,  including 
even  seemingly  improbable  ones,  are  subjected  to  full  analytic  development  and 
rigorous  testing.  Synthesis  and  analysis  are  complementary  and  in  fact  interdependent: 
by  itself,  creative  imagination  can  posit  a  wide  range  of  hypothetical  scenarios  of 
undetermined  plausibility,  but  cannot  test  or  substantiate  them  so  as  either  to  eliminate 
them  or  convert  them  into  forecasts.  This  conversion  requires  formalization.  In  this 
stage,  the  assumptions  defining  the  multiple  alternative  hypotheses  are  modeled  for 


this  paper  embraces  the  thesis  argued  by  Bazerman  and  Watkins  that  many  surprises  are  predictable  and 
should  have  come  as  no  surprise.  Bazerman,  Max  and  Michael  Watkins:  Predictable  Surprises:  the  Disasters 
You  Should  Have  Seen  Coming  and  How  to  Prevent  Them;  Harvard  Business  School  Press,  2004.  See  also 
Watkins  and  Bazerman:  "Predictable  Surprises:  the  Disasters  You  Should  Have  Seen  Coming";  Harvard 
Business  Review  (reprint),  March  2003. 

General  and  complex  systems  theory  is  another  discipline  with  a  thriving  literature.  Good 
introductions  include  Von  Bertalanffy,  Ludwig:  General  System  Theory:  Foundation,  Development, 
Applications;  Braziller,  1968;  and  Waldrop,  Mitchell  M.:  Complexity:  Life  at  the  Edge  of  Order  and  Chaos; 
Simon  &  Schuster,  1992.  See  also  SFl  and  NECSl,  fn.  16  above. 

18  The  9/11  Commission  attributed  the  failure  to  understand  the  gravity  of  fhe  looming  ferrorisf  fhreat  to 
a  range  of  causal  factors,  prominent  among  which  was  a  'failure  of  imagination'.  The  methodology 
proposed  herein  would  implement  the  commission's  call  for  fhe  1C  fo  'insfifutionalize  imaginafion' 
through  a  combination  of  alfernative  analyses,  'red-teaming'  (i.e.,  red-cell  role-playing  and  simulation 
gaming),  and  "rigorous  analytic  methods"  {9/11  Commission  Report,  op.  cit..  Chapter  11,  pp.  344-348). 

1^  'Abduction',  a  concept  generally  credited  to  Charles  Sanders  Peirce,  refers  to  the  formulation  and 
selection  of  hypotheses.  It  often  is  characterized  as  'inference  to  the  best  explanation'.  As  such,  it  is  not 
an  alternative  to  induction  and/ or  deduction.  Instead,  it  interweaves  both  in  complex  sequences  to 
derive  explanations  or  predictions. 
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internal  consistency,  extrapolated  into  the  future,  and  compared  with  known  and 
incoming  intelligence  evidence.  The  veracity  of  such  analysis  is  largely  a  function  of  the 
range  of  alternative  hypotheses  generated  in  the  preceding  stage.  What  is  required, 
then,  is  a  continuous,  total-systems  feedback  loop  between  synthetic  induction  and 
analytic  deduction.  This  organizational  learning  spiral  traditionally  has  been  known 
as  scientific  method. 


Forging  the  Solution:  Predictive  Network-Centric 
Intelligence  Assessment 

Scientific  Methodology:  Principles  and  Practices 

Scientific  methodology  marries  intuitive,  open-minded  imagination  with  skeptical 
standards  of  proof.  Archimedes  sat  in  a  bathtub  and  conceived  buoyancy,  Newton 
noted  the  falling  apple  and  conceived  universal  gravitation,  and  equally  intuitive 
inspiration  prompted  Einstein  to  ask  if  space  and  time  might  be  relative  rather  than 
absolute  phenomena.  But  all  then  subjected  their  hypotheses  to  rigorous  mathematical 
formalization  and  both  logical  and  factual  proof,  by  comparing  predictions  deducible 
from  the  formalized  theory  with  observable  empirical  data. 

In  fact,  the  entire  purpose  of  science  is  to  seek  data  and  infer  predictions 
concerning  future  outcomes  under  specified  conditions.  The  purpose  of  science  and 
the  test  of  its  effectiveness  lies  in  prediction  that  is  more  accurate,  precise  and  hence 
reliable  than  that  produced  through  any  other  method  of  investigation  and  reasoning. 

Science  achieves  this  through  the  union  of  complementary  opposites  in  both  its 
underlying  epistemology  (the  philosophy  of  knowledge)  and  in  its  methodology. 
Epistemologically,  science  unites  empiricism,  the  doctrine  that  experience  and 
observation  are  the  basic  data  source  for  physical  fact,  in  complementary 
interdependence  with  rationalism,  the  doctrine  that  logical  and  mathematical 
classification  and  inference  are  the  basic  methods  required  for  sound  interpretation  of 
observed  fact.^o  This  'rational-empirical'  epistemology  recognizes  and  exploits  the 
power  of  deductive  methods  (logic  and  mathematics)  to  order  physical  facts  into 
meaningful  patterns  (theories  and  models)  and  then  project  beyond  these  to  make 
estimates  about  facts  for  which  we  have  as  yet  no  empirical  data.  This  is  scientific 
prediction,  made  possible  only  by  the  creative  tension  between  rationalism  and 
empiricism  and  the  synergy  unleashed  by  their  symbiosis. 

To  close  the  rational-empirical  loop,  science  requires  that  all  estimates  and 
assessments  (all  hypotheses  and  theories)  be  tested,  both  rationally,  for  internal  logical 


20  The  literature  on  the  philosophy  of  science  is  voluminous.  Two  overview  works  can  be  identified  as 
particularly  influential  in  shaping  the  present  proposal:  Cromer,  Alan:  Uncommon  Sense:  the  Heretical 
Nature  of  Science;  Oxford  University  Press,  1993;  and  Losee,  John:  An  Historical  Introduction  to  the 
Philosophy  of  Science;  Oxford  University  Press,  1972/2001. 
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and/or  mathematical  coherence,  and  empirically,  for  correspondence  with  the 
observable  world.  This  testing  is  the  essence  of  Francis  Bacon's  experimental  method. 
Informative  results  either  corroborate  hypotheses  and  dictate  their  retention,  directly 
contradict  them,  thus  dictating  their  elimination,  or  (more  often)  limit  their  scope  of 
applicability,  thus  dictating  their  modification.  This  process  requires  multiple 
alternative  and  in  fact  competing  hypotheses,  since  testing,  as  noted,  is  a  process  of 
elimination. 

Scientific  Method 

'Method'  applies  methodology  to  the  solution  of  practical  problems.  In  science,  the 
problem  involves  the  quest  for  knowledge  of  the  world.^i  In  intelligence,  the  target  of 
interest  involves  the  capabilities  and  intentions  of  foreign  actors. 

Scientific  method  implements  the  aforementioned  rational-empirical  (analytico- 
synthetic)  feedback  loop:  practitioners  observe  and  measure  phenomena  of  interest  and 
then  delineate  the  relationships  among  the  data  through  formal  theory  (systems  of 
logical  and/or  mathematical  propositions  formulated  as  models  and  algorithms). 
These  theories/ models  then  support  the  deduction  of  necessary  observable 
implications:  'if  these  data  are  true,  and  this  theory/model  is  valid,  then  this  other 
observable  fact  necessarily  follows'.  Such  testable  deductions  postulate  correlation  or 
causation  in  the  form  of  'counterfactual  conditional  propositions'  or  hypotheses,  in 
which  the  baseline  data  and  theory  function  as  a  set  of  independent  variables  and  the 
expected  outcomes  are  dependent  variables.  These  hypotheses  are  tested  first  for 
logico-mathematical  coherence  and  then  typically  through  laboratory  experimentation 
in  which  independent  variables  are  manipulated  and  their  outcomes  are  recorded  and 
then  compared  with  field  observation  (e.g.,  intelligence  collection). 

In  sciences  that  study  phenomena  and  systems  that  cannot  be  manipulated 
physically  (e.g.,  the  heavens,  the  Earth,  the  past  and  mankind),  scientists  must 
substitute  laboratory  work  and  models  for  physical  experimentation  on  real-world 
specimens.  They  assemble  observable  data  from  the  historical  record  (natural  history  or 
human),  but  they  must  perform  all  of  their  experiments  on  models.  Experimentation  on 
models  is  known  as  simulation,  and  is  conducted  almost  entirely  on  computers.  This 
work  is  so  important  in  modern  science  that  an  entire  methodological  subdiscipline  has 
emerged  called  modeling  and  simulation  (M&S).^^ 


In  logic  and  scientific  method,  again  the  literature  is  vast  and  substantial.  Salient  authors  include,  in 
addition  to  Aristotle  and  Francis  Bacon,  Carl  G.  Hempel,  Rudolf  Carnap,  Hans  Reichenbach,  R.B. 
Braithewaite,  Ernst  Nagel,  Frederick  Suppe,  John  Holland,  Colin  Howson  &  Peter  Urbach,  Hugh  G. 
Gauch,  David  A.  Sebum,  Paul  Thagard,  and  Edward  Waltz  (op.  cit.).  This  proposal  has  been  guided  in 
great  part  by  the  work  of  philosophers  associated  with  logical  positivism/ empiricism,  such  as  Carl 
Hempel,  Hans  Reichenbach  and  Karl  Popper. 

22  See,  for  instance.  Cash,  John  L.:  Would-Be  Worlds:  How  Simulation  is  Changing  the  Frontiers  of  Science;  John 
Wiley  &  Sons,  1997;  Hausrath,  Alfred  H.:  Venture  Simulation  in  War,  Business  and  Politics;  McGraw-Hill, 
1971,  and  Schrage,  Michael:  Serious  Play:  How  the  World's  Best  Companies  Simulate  to  Innovate;  Harvard 
Business  School  Press,  1999. 
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Predictive  Network-Centric  Intelligence:  Practical  Procedures 
and  Infrastructure  for  Scientific  Assessment 

Thinking  within  the  'box'  or  paradigm  of  traditional  intelligence  'tradecraft'  and 
vertically  stovepiped  bureaucracy,  today's  managers  attempt  to  improve  imagination 
by  focusing  on  individual  analysts  and  exhorting  them  to  'think  outside  the  box'.  The 
social  sciences  have  shown,  however,  that  that  approach  is  likely  to  yield  but  little 
counter-intuitive  fruit. 

By  contrast,  modern  business  management  practice  has  taken  a  new  approach,  one 
that  opens  a  new  paradigm  for  broadening  perception  and  enhancing  the  likelihood  of 
discovery.  This  new  method  focuses  on  cross-functional  team  collaboration,  which 
combines  interdisciplinary,  interdepartmental  and/  or  interagency  experts  in  Integrated 
Project  Teams  (IPTs).  Under  the  leadership  of  professional  facilitators,  modern  IPTs 
use  a  rapidly  developing  suite  of  tools  and  techniques  for  the  marshalling  of  available 
data,  structured  brainstorming,  and  the  generation  of  plausible  hypotheses  concerning 
alternative  futures  and  contingencies.  Collaborative  teaming  supplies  a  double- 
barreled  countermeasure  against  groupthink:  first,  IPT  members  bring  together 
multiple  frames  of  reference,  which  skilled  facilitators  then  elicit,  fomenting  creative 
tension,  fruitful  conflict,  cognitive  dissonance,  discovery,  reconceptualization,  and 
organizational  learning. ^3 

As  noted  earlier,  much  current  emphasis  is  placed,  rightly,  on  virtual  collaboration. 
Distributed  collaboration,  however,  especially  if  asynchronous,  is  fraught  with 
implementation  difficulties,  and  can  never  be  as  dynamic,  intensive,  rich  or  fertile  as 
face-to-face  interaction.  It  can  commence  immediately  upon  warning,  while  physical 
congregation  requires  travel;  however,  the  difference  in  potential  productivity  and 
optempo  thereafter  is  dramatic. Thus  while  virtual  collaboration  must  become  the 
new  IC  'equilibrium  state',  it  should  be  punctuated  frequently  by  major  lab-based 
projects  for  the  production  of  national-level  intelligence  assessments,  especially  deep¬ 
looking  and  far-reaching  reassessments,  and  for  the  resolution  of  critical  controversies 
and  the  conduct  of  crisis-  and  combat-support  operations. 

This  in  turn  requires  an  institutional  setting  and  infrastructure  that  supplements 
bureaucratic  offices  and  stovepiped  chains  of  command  with  a  new  system  of 


23  For  more  on  collaborative  teaming,  see  Katzenbach,  Jon  R.  and  Douglas  K.  Smith:  The  Wisdom  of  Teams: 
Creating  the  High-Performance  Organization:  Harvard  Business  School  Press,  1993;  Schrage,  Michael:  Shared 
Minds:  the  New  Technologies  of  Collaboration:  Random  House,  1990;  and  Bennis,  Warren  G.  and  Patricia  W. 
Biederman:  Organizing  Genius:  the  Secrets  of  Creative  Collaboration:  Perseus  Books,  1997. 

24  Recent  research  in  team  performance  argues  for  the  continued  importance  of  proximity  for  team 
performance  maximization.  See,  inter  alia,  Fischer,  Bill  and  Andy  Boynton:  "Virtuoso  Teams";  Harvard 
Business  Review,  reprint,  July- August  2005,  pp.  116-123;  Hoegl,  Martin  and  Luigi  Proserpio:  "Team 
Proximity  and  Teamwork  in  innovative  Projects";  Research  Policy  33, 2004,  pp.  1153-1165;  and  Freechild, 
Sage:  "Team  Building  and  Team  Performance  Management", 

http:/  /  www.phoenixrisingcoaching.com/ documents/ Article-TeamBuildingandTeamPerformance.pdf, 

2004  (last  accessed  14  September  2006). 
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horizontal  integration  within  and  across  organizations  and  the  community  as  a  whole. 
The  optimal  solution  is  an  IC-wide  network  of  intelligence  assessment  laboratories. 

The  Optimal  Setting:  the  Computational  Collaboratory 

Unlike  vertically  integrated  bureaucracies,  laboratories  are  the  quintessential  'flat', 
egalitarian  organizations,  maximizing  intercommunication  while  minimizing 
managerial  friction.  Computational  collaboratories  would  be  purpose-designed  to 
bring  together  analysts  (substantive  intelligence  subject-matter  experts  or  SMEs)  and 
methodologists  for  cycles  of  synthesis  and  analysis.^^  Each  computational  collaboratory 
would  integrate  the  functions  of  an  'electronic  meeting  room',  for  collaboration,  and  a 
'math  lab',  for  computational  analysis,  and  teams  would  perform  the  two  activities  in 
iterative  succession  to  propel  a  spiral  of  discovery  and  organizational  learning. 

Since  these  computational  collaboratories  would  be  designed  to  stimulate  and 
integrate  both  creative  'right-brain'  and  critical  'left-brain'  functions,  they  should  be 
designed  in  two  'hemispheres'.  The  right  hemisphere  would  stimulate  group  dynamics 
and  supply  the  SMEs  with  extensive  access  to  classified  and  open-source  databases, 
with  an  ergonomic  layout  that  encourages  multilateral  interchange.  There,  professional 
facilitators  would  coach  the  teams  through  the  classic  'forming,  storming,  norming  and 
performing'  stages  of  team  development  while  driving  them  at  an  energetic  pace 
through  problem-framing,  brainstorming  and  'sense-making'  activities  using  the 
numerous  informal  and  semi-formal  techniques  intelligence  methodologists  and 
corporate  facilitators  have  developed.^^ 

The  resulting  mental  models,  scenarios  and  hypotheses  would  then  be  handed  over 
by  the  substantive  teams  to  their  assigned  technical  support  group  in  the  'math  lab'  or 
'left  hemisphere'  of  the  collaboratory  for  analytic  development  ('instantiation'  in 
computational  models)  and  simulation  testing.  The  technical  group  would  consist  of 
methodologists  (modelers,  experimental  designers  and  statisticians)  armed  with  the 
requisite  mathematical  tools  (e.g.,  Bayesian  statistics  and  multi-attribute  utility  analysis) 
and  simulation  models  (ideally  systems-dynamics  models). 

Experimental  results  and  findings  would  then  be  submitted  back  to  the  substantive 
analytic  teams  for  the  next  phase  of  expert  review.  In  this  stage  of  the  cycle,  facilitators 
would  help  the  experts  generate  new  hypotheses  to  replace  those  eliminated  in  testing. 
Senior  substantive  team  leaders  would  determine  when  the  cycling  should  be 


25  As  called  for  by  Ron  Johnston  in  "Integrating  Methodologists  into  Teams  of  Experts",  Studies  in 
Intelligence,  Vol.  47,  No.  1,  CIA/ CSi,  2003,  pp.  57-65;  see  also  chapter  5  in  his  Analytic  Culture  in  the  U.S. 
Intelligence  Community,  op.  cit. 

26  Examples  include  Heuer's  Analysis  of  Competing  Hypotheses  (ACH),  op.  cit.,  chapter  8,  Morgan 
Jones's  many  techniques  (Jones,  Morgan:  The  Thinker's  Toolkit:  14  Powerful  Techniques  for  Problem  Solving; 
Three  Rivers  Press,  1995.),  and  many  additional  group  techniques  discussed  in  the  literature  on  team 
facilitation. 

22  Collaborative  modeling  is  discussed  in  Vennix,  Jac:  Team  Model  Building:  Facilitating  Team  Learning 
Using  System  Dynamics;  John  Wiley  &  Sons,  1996;  and  in  Morecroft,  John  D.W.,  and  John  D.  Sterman  (eds): 
Modeling  for  Learning  Organizations;  Productivity  Press,  2000. 
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interrupted  for  summary  assessment  and  reporting,  based  either  on  diminishing 
analytic  returns  or  external  requirements. 

This  process  would  generate  an  organizational  learning  spiral  involving  initial 
inductive  synthesis,  rigorous  analysis,  and  final  comprehensive  synthesis  and 
intelligence  assessment.  In  fact,  twin  spirals  would  ensue:  one  involving  the 
substantive  intelligence  issue  under  investigation  and  of  concern  therefore  primarily  to 
the  intelligence  analysts,  and  the  other  involving  the  computational-collaborative 
methodology  itself,  which  must  constantly  undergo  scrutiny  and  improvement  by  the 
methodologists. 

A  suitable  name  for  the  national  intelligence  computational  collaboratory  might  be 
the  Intelligence  Training,  Assessment  and  Simulation  Center  (ITASC)^®. 
Organizationally,  none  of  the  substantive  analysts  need  be  ODNI  personnel;  the  line  IC 
production  centers  would  supply  these  visitors  to  the  ITASC.  ODNI,  though,  ought  to 
manage  the  ITASC  and  supply  its  permanent  staff  of  methodologists,  technicians  and 
administrators,  who  would  organize  and  coordinate  projects  and  agendas,  facilitate  the 
teams,  perform  all  M&S  and  analysis,  maintain  permanent  databases,  and  oversee  the 
drafting  of  ITASC  project  reports  (e.g.,  NIEs). 

Toward  a  National  Intelligence  Computational  Collaborative  Network 

The  ITASC  and  computational  collaborative  methodology  would  create  an 
opportunity  to  unite  the  entire  IC  into  a  single  enterprise  or  'system  of  systems' 
featuring  a  high  degree  of  'boundaryless'  horizontal  integration.  Once  the  ITASC  has 
lain  the  groundwork,  the  DNI  could  direct  and  resource  parallel  efforts  within  the  line 
agencies  and  commands.  Some  agencies  have  prototypes  already  in  place,  such  as  the 
Battle  Lab  in  the  Defense  Intelligence  Agency  (DIA)  and  the  Advanced  Analytic  Lab 
(AAL)  in  the  National  Security  Agency  (NS A).  Most,  however,  have  not  even  begun  to 
experiment  along  these  lines.  Such  intra-agency  labs  can  be  more  austere  than  the 
national  center.  Each  would  implement  inter-departmental  teaming  and  tools-based 
analyses  within  its  agency's  special  areas  of  responsibility. 

Analytic  labor  in  this  national  intelligence  collaborative  architecture  would  be 
divided  according  to  agency  charter.  All,  however,  should  function  as  a  node  in  a 
national  network  integrated  through  the  use  of  unified,  documented  scientific 
methodology  supported  by  a  unified  family  of  analytic  methods  and  a  shared  set  of 
models,  computational  tools,  and  databases  as  well  as  the  IT  standards  and  protocols 
required  to  ensure  full  interoperability,  data  exchange,  and  scientific  replication  (a 
'plug-and-play'  capability  necessary  for  competitive  analysis). 


28  A  name  inspired  by  the  Joint  Forces  Command's  Joint  Training,  Analysis  and  Simulation  Center 
(JTASC)  in  Suffolk,  VA.  By  confrasf  wifh  fhe  JTASC,  the  ITASC  would  of  course  address  foreign  fhreafs 
and  do  so  across  the  full  civil-milifary  specfrum.  Moreover,  if  would  prioritize  assessment  above 
training. 

29  The  Advanced  Research  and  Development  Activity  (ARDA)  has  already  begun  the  development  of  the 
requisite  infrastructure  for  fhe  kind  of  secure  wide-area  nefwork  this  1C  'system  of  systems'  would 
require  (not  unlike  DoD's  DARPANet).  This  program  is  the  Research  and  Development  Experimental 
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Once  in  place,  such  a  National  Intelligence  Computational  Collaborative 
Network^o  would  enable  full-spectrum  network-centric  intelligence.  ITASC-based 
collaborative  projects  for  the  production  of  major  and  enduring  DNI  assessments  could 
then  be  supplemented  by  near-real-time  global  interagency  production  of  current 
intelligence  for  strategic  and  tactical  I&W  and  crisis  and  operations  support.^i 
Community  watch  offices  could  undertake  continuous  virtual  collaboration  over  a 
secure,  real-time,  distributed  network  using  video  teleconferencing  (VTC)  and  a  unified 
set  of  IT  tools  and  data,  creating  a  National  Operational  Intelligence  Watch  Officers' 
Network  (NOIWON)  for  the  21®*  century.  The  DNI  could  mobilize  crisis  action  teams 
(CATs)  in  immediate  response  to  contingencies,  for  action  either  within  the  ITASC  or 
across  the  collaborative  net.  This  high-intensity,  high-velocity  system-of-systems  would 
divide  labor  according  to  agency  charter,  exchange  data  packets  and  outputs 
multilaterally,  and  synthesize  assessments  through  web-enabled  distributed  production 
and  dissemination.  Properly  implemented,  the  resulting  whole  would  vastly  exceed  the 
sum  of  the  parts,  compress  the  intelligence  cycle  rate,  and  improve  IC  agility  and  U.S. 
national-security  response  capability  by  an  order  of  magnitude. 


Conclusion:  the  Need  for  Scientific  Methodology 
and  Network  Centricity 

In  summation,  then,  this  paper  argues  that  in  order  to  overcome  pervasive,  long¬ 
standing  deficiencies  in  insight  and  warning,  the  national  intelligence  community  must 
integrate  horizontally  in  structure  and  function  and  adopt  advanced  methodology  that 
is  scientific  in  essence  and  intensive  in  execution.  We  need  to: 

•  Tie  the  multifarious  agencies  together  into  a  single  virtual  organization  through 
IT-enabled  data  sharing  and  advanced  analytico-synthetic  tools  and  techniques; 

•  Bring  IC  analysts  together  to  form  IPTs  in  collaboratories  that  maximize  both  the 
fertility  of  imagination  and  the  rigor  of  analysis; 

•  Maximize  rigor  by  using  computational  models,  metrics  and  databases,  and 

•  Ensure  that  team  cycles  build  on  previous  achievements  by  capturing  all  findings 
in  these  models  and  databases. 

The  implementation  challenge  will  be  threefold:  meeting  standing  production 
requirements  while  recapitalizing  the  community,  changing  from  a  guild-based  craft  to 
a  fully  modern  profession,  and  graduating  from  today's  largely  intuitive  pre-scientific 


Collaboration  network  (RDEC),  which  currently  links  42  1C  and  DoD  components  (intelligence  producers 
and  users)  into  a  unified  distributed  environment  for  fhe  festing  and  evaluafion  of  advanced  fools,  dafa- 
sharing  and  collaboration. 

30  Potential  acronym:  'NiCCNet'. 

31  The  ITASC  process  would  implement  Grabo's  recommendations  for  effecfive  i&W  (op.  cif.,  chapter  8, 
especially  concerning  probability  judgments,  pp.  145-156,  and  impediments  to  warning,  163-169). 
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'tradecraft'  to  a  formal  interdisciplinary  science  of  intelligence.^^  As  both  Thomas  Kuhn 
and  Clayton  Christensen  discuss,  such  paradigm  shifts  and  disruptive  innovations  are 
never  easy  or  uncontroversial.  Paradigm  shifts  involve  transitions  from  periods  of 
'normal  science'  operating  within  established  paradigms  to  'revolutionary  science'  that 
challenges  the  extant  paradigm.33  Disruptive  innovation  involves  a  similar  shift  from 
'sustaining'  to  'disruptive'  technologies.^^  In  both  cases,  innovation  begins  in  an 
immature  state,  its  ultimate  superiority  not  at  first  being  obvious.  Extensive  testing  and 
data  are  required  to  bring  it  to  full  fruition  and  prove  its  superiority  to  the  inevitable 
legions  of  doubters.  As  Galileo  Galilei  could  attest,  many  potentially  lucrative 
innovations  are  attacked  and  suppressed  during  this  time  of  acute  vulnerability. 

In  intelligence  reform,  however,  the  cliche  holds:  'failure  is  not  an  option'.  The 
national  security  stakes  are  too  great.  Senior  leadership  can  provide  the  requisite 
direction,  instructions  and  resources.  Middle  management  must  implement  the 
requisite  programs  and  projects.  And  all  of  us  will  be  called  upon  to  help  propel 
manifold  spirals  of  intelligence  assessment  to  produce  the  continuous  discovery  and 
learning  necessary  for  American  intelligence  to  adapt  to  the  dynamics,  threats,  and 
opportunities  of  the  21®*  century. 


32  On  the  need  to  move  from  guild  craftsmanship  to  full  professionalism,  see  Marrin,  Stephen:  Intelligence 
Analysis:  Turning  a  Craft  into  a  Profession',  International  Conference  on  Intelligence  Analysis,  May  2005; 
https://analvsis.mitre.org/proceedings/Final  Papers  Files/97  Camera  Ready  Paper.pdf  (last  accessed 
7  December  2006). 

33  Kuhn,  Thomas:  The  Structure  of  Scientific  Revolutions;  op.  cit. 

34  Christensen,  Clayton:  The  Innovator's  Dilemma:  When  New  Technologies  Cause  Great  Firms  to  Fail;  Flarvard 
Business  School  Press,  1997. 
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